## # A tibble: 1 × 1
## avg_agreement
## <dbl>
## 1 0.825
| code | n_units | n_coders | percent_agreement | krippendorff_alpha | cohen_kappa | fleiss_kappa | scott_pi | status | total_coded_as_1 | total_coded_as_0 | total_coding_instances | usage_frequency | krippendorff_interpretation | kappa_interpretation |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AI Image/Video (AI or not)_bin | 40 | 4 | 0.875 | 0.7555811 | NA | 0.7550708 | NA | Calculated | 134 | 26 | 160 | 0.8375 | Good | Good |
Focus on these codes during training - they show the most disagreement between coders:
These codes show good to excellent reliability between coders:
Agreement: 100.0% | Krippendorff α: 1.000 | Interpretation: Excellent | Units: 40 | Usage: 15.0% (24/160)
Agreement: 100.0% | Krippendorff α: 1.000 | Interpretation: Excellent | Units: 40 | Usage: 10.0% (16/160)
Agreement: 100.0% | Krippendorff α: 1.000 | Interpretation: Excellent | Units: 40 | Usage: 7.5% (12/160)
Agreement: 100.0% | Krippendorff α: 1.000 | Interpretation: Excellent | Units: 40 | Usage: 10.0% (16/160)
Agreement: 100.0% | Krippendorff α: 1.000 | Interpretation: Excellent | Units: 40 | Usage: 7.5% (12/160)
Agreement: 97.5% | Krippendorff α: 0.970 | Interpretation: Excellent | Units: 40 | Usage: 29.4% (47/160)
Agreement: 97.5% | Krippendorff α: 0.968 | Interpretation: Excellent | Units: 40 | Usage: 26.9% (43/160)
Agreement: 92.5% | Krippendorff α: 0.896 | Interpretation: Excellent | Units: 40 | Usage: 27.5% (44/160)
Agreement: 92.5% | Krippendorff α: 0.889 | Interpretation: Excellent | Units: 40 | Usage: 25.0% (40/160)
Agreement: 95.0% | Krippendorff α: 0.882 | Interpretation: Excellent | Units: 40 | Usage: 14.4% (23/160)
Agreement: 87.5% | Krippendorff α: 0.867 | Interpretation: Excellent | Units: 40 | Usage: 48.8% (78/160)
Agreement: 87.5% | Krippendorff α: 0.860 | Interpretation: Excellent | Units: 40 | Usage: 61.3% (98/160)
Agreement: 87.5% | Krippendorff α: 0.800 | Interpretation: Excellent | Units: 40 | Usage: 19.4% (31/160)
Agreement: 95.0% | Krippendorff α: 0.787 | Interpretation: Good | Units: 40 | Usage: 6.2% (10/160)
Agreement: 80.0% | Krippendorff α: 0.765 | Interpretation: Good | Units: 40 | Usage: 39.4% (63/160)
Agreement: 87.5% | Krippendorff α: 0.756 | Interpretation: Good | Units: 40 | Usage: 83.8% (134/160)
Agreement: 87.5% | Krippendorff α: 0.756 | Interpretation: Good | Units: 40 | Usage: 83.8% (134/160)
Agreement: 87.5% | Krippendorff α: 0.756 | Interpretation: Good | Units: 40 | Usage: 83.8% (134/160)
Agreement: 80.0% | Krippendorff α: 0.743 | Interpretation: Good | Units: 40 | Usage: 30.0% (48/160)
Agreement: 92.5% | Krippendorff α: 0.694 | Interpretation: Good | Units: 40 | Usage: 8.1% (13/160)
Agreement: 72.5% | Krippendorff α: 0.692 | Interpretation: Good | Units: 40 | Usage: 51.9% (83/160)
Agreement: 90.0% | Krippendorff α: 0.688 | Interpretation: Good | Units: 40 | Usage: 8.8% (14/160)
These content pieces have the most disagreements across all codes:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Coders: Doortje, Ella, Lynn
Key Variables:
Coders: Daisy, Doortje, Ella, Lynn
Key Variables:
Use this table to search for any code and see all content that was coded with that code.
Coding Coverage by Coder:
Type any code name in the search box below to find all content that was coded with that code.
Note: This table only shows content that was coded by at least 2 coders to ensure meaningful comparisons.
Dashboard generated on 2025-09-23 18:30:10.574687
Key Implementation Updates:
✅ Category Alignment Fixed: All binary conversion
logic now uses exact category text from categories.txt
✅ Conditional Coding Logic Implemented: - If coder determined image/video is “NOT AI-generated”, then ALL subsequent variables (AI Labeling, Number of People, Tonality, etc.) are correctly treated as NA (not coded) - This ensures reliability calculations only include coders who had the opportunity to code each variable
✅ AI Binary Variables Working: - “Partly AI-generated” correctly counts as 1 in “AI or not” binary variable - All AI categories (Completely, Partly, Not AI, Unsure) properly converted to binary format
Impact: - 23 responses were coded as “NOT AI-generated” - These coders are now correctly excluded from reliability calculations for subsequent variables - More accurate Krippendorff’s Alpha calculations for AI-related and content variables